cd <old-server-install-dir>/bin ./rhq-server.bat remove
This page is for upgrading an existing RHQ Installation. To install a new RHQ environment go to Installing the Server.
Upgrading RHQ includes upgrading all of the components in the overall topology; one or more RHQ Servers, the RHQ Storage Nodes making up the storage cluster, and all of the RHQ Agents. Although, RHQ Agents will auto-update if they are prepared for auto-update.
RHQ does not support mixed-versions among running components. When upgrading RHQ, all RHQ components upgraded with rhqctl must be brought down and upgraded before returning to service.
This does not include those RHQ Agents prepared for auto-update - those will not need to be shutdown. They will auto-update when they connect to the new servers.
Loss of minimal monitoring data is inevitable given the down-time involved in shutting down instances of the RHQ Agents during the upgrade process. In addition, if you have a resource in inventory corresponding to the RHQ Server itself, upgrading RHQ will entail the loss of all data for that RHQ Server resource.
Perform the following steps prior to your upgrade. The server components (RHQ Servers and Storage Nodes) should still be running until the step instructing to shut them down.
Please review all of the information at Windows Installation prior to upgrade on Windows.
RHQ Agents monitoring your RHQ Servers and RHQ Storage Nodes should be stopped prior to upgrade. Unlike remote agents, they are upgraded as part of the rhqctl upgrade command itself.
Stop the agents via the rhqctl stop --agent command.
Wait for the agents to fully stop before continuing. Note that remote agents will typically auto-update upon connection with an upgraded server, unless configured otherwise.
Versions prior to RHQ 4.8 did not have the rhqctl utility. If an RHQ Agent is running locally, monitoring the RHQ Server, it must be stopped prior to upgrade:
Stop a pre RHQ 4.8 Agent via the agent wrapper's stop command, the quit prompt command, or the RHQ Agent resource operation.
When upgrading, use the --from-agent-dir option to the "rhqctl upgrade" command.
It is most efficient to remove "out of service" platform resources prior to the upgrade. If you have machines you have stopped, decommissioned or will stop monitoring then stop any RHQ Agents that are still running on those machines. Then, via the RHQ GUI, uninventory the platforms that are no longer relevant.
Because the upgrade will replace the RHQ Servers, any currently imported RHQ Server resources will no longer be available after the upgrade. New RHQ Server resources will be discovered and can be subsequently imported.
If, for some reason, you prefer to keep the old RHQ Server resources then ignore this step. They will have DOWN availability after the upgrade but can be uninventoried later, if so desired.
Note that all RHQ Agents monitoring RHQ Servers and RHQ Storage Nodes should already have been stopped.
Stop the RHQ Servers.
If you are running a pre RHQ 4.8 RHQ Server as a Windows service, uninstall the Windows service:
cd <old-server-install-dir>/bin ./rhq-server.bat remove
Stop the RHQ Storage Nodes after the servers are down.
This is only relevant to RHQ 4.8 and later. Note that rhqctl stop will stop all RHQ components previously installed with rhqctl which includes RHQ Server, RHQ Storage Node, and RHQ Agent installations.
Do not stop the RHQ Database - ensure it is still running.
Do not delete the existing RHQ Installation directory, it is used during the upgrade.
The rhqctl upgrade will merge the old rhq-server.properties into the new rhq-server.properties so you probably will not need to worry about merging them yourself. However, you will probably want to make sure most, if not all, values you had in the old properties file are the same in the new file after the upgrade.
In the rare case that you customized the old bin/rhq-server.sh (if on UNIX/Linux) or bin/wrapper/rhq-server-wrapper.conf (if on Windows) files, back them up because you will need to reapply those changes manually.
You should strongly consider backing up your database prior to proceeding. If problems arise during the database upgrade, having a backup will allow you to restore to your previous state. Not doing so risks a potential loss of all prior data.
Unzip the RHQ Distribution to the directory within which it will be executed from, e.g.:
cd /opt/rhq unzip rhq-server-4.13.0.zip
The new installation should not be copied on top of a previous installation. It is recommended to unzip to the same parent directory (e.g. /opt/rhq), making it a sibling root directory to the the existing install. All RHQ distribution zip files have a directory structure that ensure their content is extracted into a version-specific directory (e.g. "rhq-server-4.13.0") so this usually isn't a problem unless you are attempting to re-install the same version as was previously installed. For example, the above commands will create a directory named /opt/rhq/rhq-server-4.13.0
Note that the same RHQ Distribution is used to install a standalone RHQ Server, a standalone RHQ Storage Node, or a co-located RHQ Server and Storage Node. The RHQ Agent will be installed as well, as needed, from this same RHQ Distribution.
An RHQ Installation can include multiple RHQ Servers for High-Availability. And the RHQ Storage Cluster can include multiple RHQ Storage Nodes. These will be spread out on several machines, and each machine must be upgraded.
Because the upgrade will merge old settings into the new environment it should not be necessary to alter any properties in the newly unzipped distribution.
If for some reason you do need to edit rhq-server.properties prior to the upgrade:
Do not set rhq.autoinstall.database to the value of "overwrite" since this will purge all the RHQ data in the database and effectively make this a new, clean install and not a true upgrade. Note that the rhqctl control script will attempt to set this value to "auto" so an overwrite is not mistakenly performed.
Do make sure you set rhq.server.high-availability.name to the original server's name - that is, this should be the same server name of the server you are upgrading. If the previous installed server left this blank, you can leave this blank as well, but you must ensure that the hostname/domain of the machine has not changed. If that did change, you must explicitly put the old name of the server as the value of this property. If this name does not match the previous server's name, you will effectively be adding another server to the RHQ HA Cloud, which is not what you want to do when simply upgrading an already existing server. Again, rhqctl will attempt to merge the old value in the old rhq-server.properties file to the new rhq-server.properties file, so you really should not have to do anything.
If you aren't sure what the name was, you can ask the server installer to give you a list of all existing servers currently registered in the RHQ system. You do this via the --listservers (aka -l) command line option to the rhq-installer script found in your currently installed server location:
<rhq-install-dir>/bin/internal/rhq-installer.[sh,bat] --listservers
Starting with RHQ 4.13 it is required for standalone Storage Nodes that rhq-server.properties be configured with correct database properties. This enables the storage nodes to correctly register with the database. When upgrading a standalone Storage Node from RHQ 4.12 or earlier, edit the rhq-server.properties (the version in the --from-server-dir location), as needed, to ensure following properties are valid:
rhq.server.database.connection-url
rhq.server.database.user-name
rhq.server.database.password
The storage node in 4.8 included native libraries for different things like data compression. If your platform has support for those native libraries, then compression was used by default on the data files. In 4.9, the native libraries have been removed in an effort to provide better cross-platform support. Compression has to be turned off in 4.8 before upgrading; otherwise, the storage node will not run in 4.9. Download the patch zip file here. Unzip the file and follow the instructions in the .sh/.bat file.
The patch script runs the CQL Shell for Apache Cassandra cqlsh which requires python 2.5, 2.6 or 2.7 to run. The script also runs nodetool which requires JRE 1.7 set as the JAVA_HOME.
There are some important things to note when upgrading pre RHQ 4.8 servers. In particular, the upgrade will involve the initial installation of an RHQ Storage Node (unless --use-remote-storage-node is specified). And every RHQ Storage Node requires an RHQ Agent to manage it. In addition, if you wish to retain your old measurement data, you will need to run the data migrator tool to transfer the old measurement data from your RDBMS database to the new storage backend.
Starting in RHQ 4.8 the RHQ Agent will be installed for all servers co-located with an RHQ Storage Node, which is the default. The Agent will be installed to a fixed directory, which is the parent directory of the new server installation (making it a sibling directory).
For Pre RHQ 4.8 upgrades use the --from-agent-dir option to specify the old agent location, if it exists. For example, if you are upgrading an older agent located at /home/rhqproject/rhq-agent, you would need to specify the option "--from-agent-dir /home/rhqproject/rhq-agent" in order to upgrade that agent. The upgraded agent will be migrated to the new, fixed location, mentioned above.
Use the --from-agent-dir option even if the agent exists in what will become the fixed location.
To alter any defaults for the RHQ Storage Node create a <rhq-install-dir>/bin/rhq-storage.properties file. In this file you can specify any of the RHQ Storage Node --storage-config options. These include the directories for data storage, host and port information, and several other options.
While optional, if you do not run the data migrator tool, all of the measurement data from the old install will no longer be available. If you wish to retain your historical measurement data, you need to run the data migrator tool. The data migrator tool is a command line tool that is run after the other upgrade steps have been completed. It can generate an estimate of how long data migration would take or run the data migration manually. Or the legacy data can be abandoned by simply not running the data migration.
To run the data migrator after the upgrade is complete, see the tool's help:
> ./rhq-data-migration.sh --help
On Windows the RHQ Server, RHQ Storage Node, and RHQ Agent will be installed as Windows Services. See more here before installing.
Because the upgrade may result in new Windows services, it is important to ensure the services will be run as the desired user. Set the required environment variables as needed.
The RHQ upgrade is performed via the <rhq-install-dir>/bin/rhqctl upgrade command. Please review the command options before continuing - some options may apply to your environment.
You will want to run this upgrade command to upgrade any standalone Storage Nodes first, then Servers after (as described in the following sections).
In particular the --from-server-dir option is required. It specifies the RHQ install directory you are upgrading from and it is from there that settings will be merged. For example, if your old server was installed at /opt/rhq/rhq-server-4.9, you will want to use the option "--from-server-dir /opt/rhq/rhq-server-4.9"
The basic upgrade command looks like:
> cd <new-install-dir>/bin > ./rhqctl upgrade --from-server-dir <old-install-dir> [--from-agent-dir <old-agent-dir>] [options]
This will:
Merge the old RHQ Storage Node settings into the RHQ Storage Node settings, or install a new RHQ Storage Node, if necessary.
Merge the old RHQ Server settings into the new RHQ Server settings.
Update the old RHQ Agent or install a new RHQ Agent if necessary.
During the upgrade you may see error messages similar to the following in the console. These error messages are harmless.
ERROR [ClientCommandSenderTask] {ClientCommandSenderTask.send-failed}Failed to send
command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.timeout=1000,
rhq.send-throttle=true}]; params=[{targetInterfaceName=org.rhq.enterprise.communications.Ping,
invocation=NameBasedInvocation[ping]}]]. Cause: org.jboss.remoting.CannotConnectException:[.....]
The RHQ Storage Cluster includes one or more RHQ Storage Node installations. An RHQ Storage Node may be co-located with an RHQ Server, or it may be standalone. For each standalone RHQ Storage Node installation perform the upgrade command. Note that co-located RHQ Storage Nodes will be upgraded when those RHQ Server installations are upgraded in the next step.
An RHQ Server Cluster includes one or more RHQ Server installations. An RHQ Server may be co-located with an RHQ Storage Node, or it may be standalone. Perform the upgrade command on each RHQ Server installation. Note that for standalone servers, the --use-remote-storage-node=true option is required.
Note that running multiple RHQ Servers is known as an HA (High-Availability) installation.
When an RHQ Server upgrade completes it prints out a summary of the installed topology and the currently installed versions of each RHQ Server and RHQ Storage Node. This can help you see which nodes remain to be upgraded. Note that the same summary can be generated manually by executing rhqctl upgrade --list-versions from an RHQ Server installation (not available from a standalone RHQ Storage Node installation).
Note that the report will only include those Storage Nodes that existed prior to the upgrade. Any Storage Nodes added as part of the upgrade process will not show up in the report. For example, when upgrading from a pre-Storage-Node version, no newly added storage nodes will be included in the report.
Not every upgrade release will contain RHQ Storage Cluster Schema changes. But if it does this step must be performed after the upgrades of the RHQ Storage Node and RHQ Servers. The upgrade summary generated at the end of an RHQ Server upgrade, or manually when executing rhqctl upgrade --list-versions from an RHQ Server installation, will tell you whether you need to perform the RHQ Storage Cluster Schema update step.
When upgrading from a pre-Storage-Node version you must execute this step to install the initial schema!
If so:
Complete Server and Storage Node upgrades.
Start all Storage Nodes to prepare them for a schema update.
Use rhqctl start --storage.
Servers and Agents should not be running.
Update the Storage Cluster via rhqctl upgrade --storage-schema.
This can be run from any RHQ Server installation (not from a standalone Storage Node installation).
This only needs to be run one time, the schema will be updated for the entire RHQ Storage Cluster.
All of the RHQ Storage Nodes in the RHQ Storage Cluster must have been upgraded and must be running in order to process the schema changes. This command can take time, perhaps hours, based on the amount of existing data and the nature of the schema changes.
Skip this step if all of the RHQ Agents are configured for auto-update. Otherwise, follow the Manually Upgrading the Agent instructions to manually upgrade relevant RHQ Agents.
Performing rhqctl upgrade will not leave the upgraded services running. It is now safe to start the upgraded components via rhqctl start, or for remote agents, in their standard fashion.
Since RHQ 4.13 the --start has been deprecated and is ignored. The convenience of an automatic start option was outweighed by problems caused trying to start a component prior to the entire RHQ Installation being completely upgraded. For a more predictable upgrade experience the rhqctl start command must be invoked to start components after the upgrade steps are complete.